Pueblo/MXP HTML entity decoding (+ quote-aware tag scan) - #6
Merged
Conversation
In Pueblo HTML mode the server entity-encodes reserved characters (a literal '>' arrives as ">"), so the client must decode them on render — previously entities were shown verbatim. AnsiParser now decodes entities, but only inside markup mode (Pueblo active, or MXP active and not Locked); in plain ANSI output '&' stays literal. Decoder (per the Pueblo reference client + BeipMU behaviour): - named: < > & " ' © ® ™ (case-insensitive) - numeric &#NN; decimal and &#xNN; hex, decoded to full Unicode (we're UTF-8) - control-range (< U+0020), out-of-range and surrogate code points are ignored - a terminating ';' is required; a lone or unrecognised '&' is emitted literally and never dropped - attribute values (xch_cmd/href/hint/...) are entity-decoded too Also made the tag-terminator scan quote-aware so a literal '>' inside a quoted attribute value (e.g. <a xch_cmd="say 5 > 3">) no longer truncates the tag. Tests: PuebloEntityTests (10) cover core/apos/nbsp, decimal+hex numeric, control ignore, lone/unknown '&' pass-through, markup-gating, MXP decoding, attribute decoding, and the raw-'>'-in-attribute case. Full suite 192/192, 0 warnings. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HHCRc5CJ6595iMzEgYYdQp
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Pueblo HTML mode entity-encodes reserved characters — a literal
>arrives from the server as>. We were rendering entities verbatim.AnsiParsernow decodes HTML entities, only inside markup mode (Pueblo active, or MXP active and not Locked); in plain ANSI output&stays literal.Based on the Pueblo reference client (
uecasm/pueblo) and BeipMU (BeipDev/BeipMU,src/TextToLine.cpp) entity/tag behavior.Decoder
< > & " ' © ® ™(case-insensitive — friendlier than the reference client).&#NN;decimal and&#xNN;hex → full Unicode (we're UTF-8, so no 8-bit cast).;is required; a lone or unrecognized&is emitted literally and never dropped; control-range (< U+0020), out-of-range, and surrogate code points are ignored (per MXP).xch_cmd/href/hint/ … values are entity-decoded too.Also
Made the tag-terminator scan quote-aware, so a literal
>inside a quoted attribute value (e.g.<a xch_cmd="say 5 > 3">) no longer truncates the tag — a sturdiness guard the Pueblo reference client also has.Tests
PuebloEntityTests(10): core entities, apos/nbsp, decimal+hex numeric, control-range ignore, lone/unknown&pass-through, markup-gating (no decode in plain mode), MXP decoding, attribute decoding, and raw->-in-attribute. Full suite 192/192, 0 warnings.Independent of #5 (this branch is off
master; #5 doesn't touchAnsiParser).🤖 Generated with Claude Code